Solution Dynamics, Causality, and Critical Behavior of the Regularization Parameter in Total Variation Denoising Problems

We analyze the role of the regularization parameter λ in the total variation (TV) denoising model. There are several contributions in this paper. (1) We realize that, beyond controlling the smoothness of the solution, λ exhibits another significant behavior — causality. This property allows us to solve the problem by incrementally increasing λ so that each solution at a given regularization parameter can be used to solve the problem at a larger parameter efficiently. We call such a technique parameter marching. (2) While λ is allowed to take a continuum of non-negative values, we show that only a unique finite number of them are critical and useful in the sense that they correspond to meaningful changes in signal features. Furthermore, we present the construction of these critical λ’s. (3) Since the analysis of the TV model is carried out by deriving exact solutions and investigating the dynamics of the solutions as λ varies, many properties of the model which previously have not been studied to a rigorous extent are clearly revealed. A discussion of other possible further research within the parameter marching framework is also given. In particular, we present some insights into the improvement of the TV model provided by our analysis. We also discuss the possibility of applying parameter marching techniques to solving general Tikhonov-type regularization problems.